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Abstract-  Mautly  Pathfinder  is  a  system  designed  to  obtain 
images  of  the  surrounding  world  presented  to  a  blind  person  by 
any  video  camera  and  to  manage  those  images  to  obtain  only  the 
significant  information  for  the  user  such  as  the  shape,  size, 
position  and  relative  movements  of  possible  hazards.  This 
information  is  transmitted  to  the  user  by  a  vibrating  matrix 
located  on  the  abdominal  region  of  the  body.  Furthermore, 
pulses  in  which  frequency  varies  depending  on  distance  between 
such  hazards  and  the  user  are  sent  to  a  vibrating  interface 
located  on  the  chest  of  the  impaired  person.  The  purpose  is  to 
give  enough  information  to  the  user  to  create  a  cognitive  map  of 
the  local  environment.  The  system  is  designed  to  be  wearable 
and  to  work  at  real  time,  which  means  that  the  data  processing 
and  the  results  delivering  of  such  information  are  done  almost 
instantly. 

I.  INTRODUCTION 

Recent  researches  have  shown  that  Electronic  Aids  for  visual 
impaired  persons  must  provide  to  the  user  only  small 
amounts  of  information  in  order  to  benefit  the  mobility 
performance  without  interfering  with  the  information  already 
gathered  through  other  senses  and  aids  like  canes [8]  or 
dogs [9].  In  other  words,  Electronic  Aids  must  be  thought  as  a 
secondary  mobility  aid.  This  means  that  Electronic  Aids 
should  work  together  with  primary  aids  (canes  and  dogs). 

Researches  performed  by  the  psychologist  Alfred  Leonard, 
the  founder  of  the  BMRU  (Blind  Mobility  Research) [1]  have 
shown  that  small  amounts  of  relevant  information  can  turn 
the  long  cane  performance  into  a  smooth,  graceful  and  safe 
performance.  Leonard  describes  three  basic  problems  found 
on  the  use  of  primary  aids:  orientation  with  respect  to  the 
general  environment,  orientation  with  respect  to  the  local 
environment  and  the  detection  of  hazards  not  detected  by  use 
of  the  primary  mobility  aids. 

Therefore,  Mautly  Pathfinder  has  been  designed  to  give 
information  to  the  user  about  the  hazards  located  in  front  of 
him/her.  The  user  will  be  capable  of  creating  a  cognitive 
map [2]  of  the  local  environment  by  estimating  the  sizes, 
positions  and  distances  of  hazards  located  on  his/her 
surroundings.  Also  the  visual  impaired[4]  user  will  be 
capable  to  know  if  hazards  are  moving  and  will  know  the 
path  of  those  movements  and  the  speed.  Of  course  the 
acquisition  of  these  kind  of  data  have  to  be  improved  by 
training. 

The  system  consist  of  two  subsystems:  video  subsystem  and 
an  ultrasonic  subsystem.  The  first  one  uses  a  video  camera 
which  takes  images  of  the  world  presented  to  the  visual 
impaired  person[4].  The  processed  images  are  delivered  to 
the  user  through  a  vibrating  matrix  located  at  the  abdominal 


Fig.  Example  of  how  one  image  is  sampled. 


region  of  the  body.  The  matrix  has  been  built  with  6x4 
motors  and  those  motors  will  be  turned  on  and  off 
depending  on  the  results  of  the  image  processing.  This 
means  that  the  video  resolution  will  be  reduced  to  6x4  black 
and  white  where  each  pixel  will  correspond  to  one  motor.  In 
the  other  hand,  the  ultrasonic  system  is  composed  of  an 
ultrasonic  device  which  purpose  is  to  measure  the  distance 
between  the  user  and  the  hazards  located  in  front  of  him/her. 
This  measured  distance  is  used  to  vary  the  frequency  of  a 
square  signal  that  feeds  another  motor  (vibrating  interface) 
located  now  on  the  chest.  So  depending  on  the  operation 
frequency  of  this  motor,  the  impaired  person  will  know  the 
distance  between  him/her  and  the  hazard.  Furthermore  with 
the  use  of  the  video  subsystem  he/she  could  know  the  size 
and  position  of  the  hazard  on  the  local  surrounding  and  also 
the  speed  at  which  he/she  will  reach  this  hazard  or  the  hazard 
will  reach  him/her  in  case  that  the  hazard  moves  toward 
him/her.  Notice  that  if  the  hazard  presents  movements  to  the 
right,  left,  up  or  down,  the  user  will  notice  them  by  the  use  of 
the  video  system  only.  But  if  the  hazard  presents  movements 
frontward  or  backward,  they  will  be  detected  by  the  use  of 
both  subsystems  (video  and  ultrasonic  systems). 

II.  METHODOLOGY 

Mautly  Pathfinder  is  a  system  controlled  by  a 
microcontroller  [5]  and  a  PLD  (programmable  logical 
device)  [7].  The  system  is  composed  by  an  ultrasonic 
sensor  [6],  which  is  in  charge  of  measure  the  distance  between 
the  impaired  person  and  the  hazard  located  in  front  of 
him/her,  and  a  video  camera,  which  is  capable  of  delivering  a 
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video  output  with  NTSC[3]  (National  Television  Systems 
Committee)  standard.  The  video  information  is  treated  by  the 
PLD[7]  in  such  a  way  that  the  resolution  of  images  are 
reduced  to  a  matrix  of  6x4  black/white  to  be  represented  on  a 
matrix  of  motors,  which  will  work  as  a  vibrating  matrix 
located  on  the  abdominal  region  of  the  body.  Meanwhile  the 
distance  information  is  treated  by  a  microcontroller [5]  that 
transforms  the  information  into  frequency  variations  that  turn 
off  and  on  the  ultrasonic  interface  (motor)  in  a  faster  or 
slower  way.  This  interface  is  located  on  the  chest. 

Ultrasonic  system: 

The  idea  of  the  ultrasonic  system  is  to  measure  the  distance 
between  the  impaired  person  and  any  hazard  located  in  front 
of  him/her,  so  that  the  person  can  estimate  how  far  an  object 
is.  Therefore  the  use  of  one  motor  that  works  at  specific  time 
intervals  will  be  the  interface  for  the  ultrasonic  system.  This 
motor  is  located  on  the  chest  of  the  impaired  person  and  the 
time  intervals  will  depend  on  the  distance  measured  by  the 
sensor. 

This  ultrasonic  sensor[6]  after  measuring  the  distance  is 
capable  of  sending  the  data  through  a  serial  communication 
port  that  uses  the  standard  of  communications  RS-232.  Then 
a  microcontroller^]  is  used  to  receive  the  distance  measured 
by  the  sensor  and  depending  on  the  data  it  modifies  the 
timing  of  the  intervals  for  turning  on  and  off  the  DC  motor, 
being  shorter  for  closer  objects. 

The  microcontroller [5]  sends  the  character  "M"  (0x4D)  to  the 
ultrasonic  sensor[6]  via  the  serial  communication  port.  Then 


the  ultrasonic  sensor  performs  the  measuring  and  returns  5 
characters  corresponding  to  the  measured  distance.  After  the 
5  characters  have  been  received,  the  microcontroller [5] 
changes  the  hexadecimal  representation  of  each  character  to 
their  decimal  representation.  Then  a  mathematical  operation 
using  the  decimal  representation  of  each  character  is 
performed  to  obtain  a  float  number.  The  resulting  float 
number  is  multiplied  by  17  and  divided  by  10000  so  that  the 
result  obtained  is  the  distance  in  meters.  This  number  is  used 
to  calculate  a  time  delay  which  varies  the  frequency  of  the 
pulses  applied  to  the  motor.  The  frequency  depends  linearly 
of  the  measured  distance. 

Video  System: 

This  part  of  the  system  manages  the  input  video  signal 
received  from  the  video  camera  in  such  way  that  the 
necessary  information  for  the  reduction  of  resolution  can  be 
pulled  out.  The  main  idea  is  to  reduce  the  resolution  of  the 
video  frames  to  6x4  black/white.  Thus  the  video  can  be 
managed  through  the  motor  matrix  located  at  the  abdominal 
part  of  the  body  of  the  impaired  person. 

Firstly,  the  video  signal  received  from  the  video  camera  is 
treated  by  a  video  sync  separator,  which  separate  the 
horizontal/vertical  synchronization  [3]  and  odd/even 
components [3].  These  components  are  used  to  synchronize 
the  system  clock  to  the  video  signal,  so  that  if  the  video 
signal  suffers  an  alteration,  the  system  will  be  able  to  follow 
the  alteration  and  recover  the  synchronization  again. 

The  horizontal  synchronization  signal  activates  one  oscillator, 
which  works  as  the  system  clock.  When  the  horizontal 
synchronization  signal’s  state  is  “1”,  the  oscillator  is 
activated  and  during  the  clock  cycles,  the  video  signal  begins 
to  be  sampled.  When  the  end  of  one  line  of  one  image  is 
reached,  the  oscillator  is  deactivated  and  the  sampling  of 
video  signal  is  stopped.  Subsequently,  at  the  beginning  of  the 
next  line  the  clock  is  activated  again.  In  this  way,  the  system 
can  be  synchronized  to  the  video  signal  to  guarantee  that  the 
sampled  data  belongs  to  the  video  information. 

To  activate  the  system  clock  one  BJT  transistor  (NPN)  is 
being  used  as  a  switch.  The  oscillator  is  a  logic  inverter 
arrange  connected  through  an  RC  circuit.  The  clock’s 
operation  frequency  is  given  by: 

f=1.23RC 

where  f=1.03MHz  which  is  equal  to  a  video  line  /  58.  That 
means  that  58  samples  are  taken  out  from  one  video  line. 

During  each  clock’s  pulse  one  sample  is  taken  out  from  the 
video  signal  to  determine  if  it  will  be  consider  as  a  black  or 
white  point. 

So,  using  the  horizontal  synchronization  signal  the  PLD 
program[7]  is  able  to  know  which  line  is  being  sampled  and 
also  the  system  clock  is  synchronized  to  the  video  signal. 

On  the  other  hand  the  odd/even  signal[3]  indicates  the  field 
that  is  being  sampled  depending  on  the  logic  state  of  this 


Fig.  Internal  structure  of  Control  System  and  User  interface. 

signal.  Considering  NTSC  format [3],  the  information  of  one 
image  is  sent  using  two  fields.  The  first  field  starts  form  the 
left  upper  corner  (line  1)  and  ends  at  the  middle  of  line  263 
(lower  part  of  image).  The  second  field  starts  from  the  half  of 
line  263  (upper  part  of  image)  and  ends  at  the  lower  right 
corner  (line  255).  On  this  way,  when  the  information  of  field 
one  is  being  sent,  the  odd/even  signal  remains  on  high  state 
and  while  the  second  field  information  is  being  sent,  it 
remains  on  low  state. 

Therefore,  this  signal  is  being  used  as  a  reset  signal  for  the 
video  system.  When  a  high  state  remains  (receiving  field  1 
information)  the  PLD  program  runs [7].  But  when  the 
odd/even  signal  [3]  state  changes  (receiving  field  2 
information)  the  system  is  reset  and  remains  like  that  until  the 
state  of  odd/even  signal  change  again. 

The  field  No.  1  as  the  field  No.  2  goes  over  the  whole  image. 
Therefore,  the  PLD  only  consider  the  data  belonging  to  the 
field  No.l  cause  the  final  output  resolution  is  very  low  (6x4). 
So  there  is  not  need  to  sample  both  fields. 

Each  sample  taken  from  the  video  signal  on  each  clock  pulse 
is  evaluated  through  a  basic  comparator  that  is  able  to 
compare  high  speed  signals.  A  fixed  reference  voltage  is 
being  used  to  compare  the  input  video  signal.  So  that,  a  video 
camera  with  automatic  brightness  control  must  be  used  to 
compensate  the  possible  brightness  variations  on  the  images. 
The  comparator  output  indicates  if  the  sample  taken  from 
video  information  will  be  considered  as  a  white  or  black 
point. 

All  the  signals  mentioned  before  are  used  by  the  PLD  to 
reduce  the  video  resolution  to  6x4  black/white. 

The  program  wrote  for  the  PLD  uses  the  VHDL  language 
[7]  (Very  High  Speed  Integrated  Circuit  (VHSIC)  Hardware 
Description  Language).  This  is  a  modular  high-level  language 
that  allows  making  a  hardware  description. 


Firstly,  the  program  describes  how  many  inputs  and  outputs 
the  system  will  require.  Totally,  there  are  24  outputs,  which 
will  activate  the  motors  placed  on  the  6x4  matrix.  The  state  of 
these  outputs  will  depend  on  the  quantity  of  black  points 
detected  at  the  corresponding  image  zone  (one  still  image 
will  be  separated  onto  6x4  zones  to  be  treated). 

There  are  4  inputs  which  are  connected  to  the  video  signals 
resulting  from  the  video  decomposition.  The  connection  of 
these  signals  with  the  corresponding  inputs  are  described 
next: 

??  The  input  "elk"  is  connected  to  the  clock  signal 
resulting  from  the  oscillator  circuit. 

??  The  input  "cmp"  is  connected  to  the  comparator 
output,  which  is  sampling  the  video  signal 
determining  whether  it  is  a  white  or  black  point. 

??  The  input  "rsth"  is  connected  to  the  horizontal 
synchronization  signal  to  count  the  lines  that  will  be 
sampled. 

??  The  input  "rstv"  is  connected  to  the  odd/even  signal 
to  know  from  which  field  the  information  is  being 
received. 

After  declaring  the  outputs  and  inputs  of  our  system,  eight 
variables  have  to  be  created  so  that  the  image  can  be 
evaluated  using  zones.  The  variable  Count  gives  information 
about  the  point  that  will  be  sampled  into  a  line.  The  variable 
line  counts  the  number  of  lines  that  will  be  sampled.  While 
the  variables  Cl  to  C6  keeps  the  number  of  black  points 
detected  in  each  image  zone.  We  have  to  remember  that  the 
final  resolution  is  6x4,  therefore  each  variable  Cx  keeps  track 
of  black  points  found  in  the  corresponding  column  of  the 
matrix.  Then,  the  system  remains  on  reset  while  the  signal 
rstv  remains  low  (information  of  field  2  is  being  sent).  When 
the  state  of  this  signal  changes  to  high  (information  of  field 
1),  the  PLD  start  working  and  the  system  starts  sampling  the 
video  signal  at  each  clock  pulse,  meanwhile  the  variable 
count  is  being  increased.  If  any  black  point  is  detected  during 
sampling,  the  corresponding  Cx  variable  is  increased  by  one. 
The  variable  count  is  used  to  know  which  Cx  variable  will  be 
increased;  Cl  is  incremented  for  values  between  1  and  13  of 
count  variable,  C2  is  increased  between  14  and  22,  C3  is 
increased  between  23  and  3 1 ,  C4  between  32  and  40,  C5  for 
count  between  41  and  49  and  finally  C6  between  count  values 
of  50  and  58. 

At  the  end  of  the  sampling  of  one  line,  the  variable  rsth 
change  its  state  to  low,  clearing  the  variable  count  to  start 
with  the  sampling  of  the  next  line.  That  means  that  the  line 
variable  is  increase  by  one  meanwhile  count  variable  is  set  to 
zero.  When  the  variable  rsth  returns  to  high  state  at  the 
beginning  of  the  next  line,  the  system  starts  sampling  the  next 
line  and  so  on  for  the  next  lines  until  the  line  63  is  reached. 
At  this  moment  the  Cx  variables  are  cleared  so  that  the  next 
six  image  zones  can  be  sampled.  The  same  process  is 
repeated  between  line  64  and  133,  line  134  and  199  and  at 
last  line  200  and  264.  That  means  that  on  line  63,133  and  199 
a  block  change  is  generated. 


While  variables  Cx  are  being  updated,  depending  if  black 
points  are  being  found  or  not,  the  value  of  these  variables  is 
checked  at  each  clock  pulse  to  know  if  they  are  higher  than 
330.  In  case  of  being  higher,  the  corresponding  output  state 
will  change  to  high.  Each  block  of  the  6x4  matrix  has  660 
points  in  it,  so  if  more  than  half  of  points  are  black,  the  block 
will  be  consider  as  a  black  block.  For  this  reason  if  the  block, 
suppose  Cl-blockl  has  more  than  330  black  points,  the  BO 
output  corresponding  to  motor  1  (1,1)  of  6x4  matrix  will  be 
activated,  provoking  that  the  impaired  person  notice  that  a 
dark  object  is  located  at  this  image  zone.  In  the  opposite  case, 
if  the  block  have  less  than  330  black  point,  the  motor  1  (1,1) 
remains  off,  provoking  that  the  impaired  person  notices  that  a 
light  object  is  located  at  this  image’s  zone.  The  same  goes  for 
the  C2-Blockl,  the  motor2  (1,2)  of  the  matrix  will  be  turned 
off  or  on  depending  on  the  number  of  black  points  located  at 
the  image’s  zone.  And  so  on  for  the  rest  of  zones. 

Finally,  when  all  the  zones  have  been  sampled,  the  input  rstv 
state  will  change  to  low,  resetting  the  system  so  that  the  field 
one  can  be  sampled  again. 


Cl 

BJocKl 

ca 

Blockl 

C3 

Hocki 

C4 

Block! 

C5 

Blacla 

CO 

Bbcki 

Cl 

&0XK2 

C2 

B!«k2 

C3 

6J0CK 

c* 

C5 

Block  3 

CO 

BlOCt  ? 

Cl 

BlOCk  3 

C2 

Block  3 

C3 

£Jock3 

C4 

Block  3 

CS 

Blocks 

06 

E-bck  3 

Cl 

Black -4 

C£ 

m-zk  4 

C 3 

Block  4 

C* 

Blctk  4 

C5 

Block  4 

C6 

Black  4 

oouht  _ 

(] 

Fig.  Simplified  Image  Matrix 


m  result 

The  system  is  still  a  prototype,  that  means,  that  better  and 
smaller  devices  can  be  used  in  order  to  reduce  size  and 
weight  of  the  entire  system.  Nevertheless,  the  price  of  the 
system  can  increase  dramatically.  But  the  fact  is  that  the 
system  is  now  totally  portable  and  can  be  reduced  more  in 
size  and  weight  with  low  power  consumption.  Also  notice 
that  the  electronic  control  is  totally  independent,  that  means 
that  no  computer  is  needed  to  be  connected  to  the  system. 
The  use  of  a  12V  battery  let  the  system  to  be  totally  portable 
and  practical,  and  the  battery  can  be  recharged  at  night. 

The  ultrasonic  device  used  now  is  not  the  optimal  one  due  to 
the  dimensions  of  hardware.  Also  the  video  camera  is  too  big 
to  carry  for  a  long  time.  But  rememb  er  that  the  system  have 
been  built  as  a  prototype  therefore  any  video  camera  with 
NTSC  video  output  and  auto-brightness  control  can  be  used 
giving  a  wide  range  of  selection.  The  ultrasonic  device  can  be 
changed  for  a  smaller  one,  preferment  from  Senix 
Corporation.  The  only  requirement  is  that  the  data  must  be 
transferred  via  serial  port  working  with  RS232  standard.  So 
the  system  is  flexible  on  these  points. 


V.  Conclusion 

The  basic  characteristics  of  secondary  aids  are  covered  by 
the  system,  it  can  be  used  together  with  primary  aids:  canes 
and  dogs,  giving  the  user  the  freedom  to  use  of  his/her  other 
senses.  The  audio  sense  is  not  interfered  by  the  system  as 
well  as  the  use  of  both  hands  which  are  the  most  important 
tools  for  a  visual  impaired  person.  The  user  would  be  able  to 
combine  and  use  the  information  gathered  from  primary  and 
secondary  aids  without  being  redundant.  Obtaining 
information  about  hazards  on  short  and  long  distances  so  that 
cognitive  maps  could  be  easily  created.  But  as  everything, 
training  on  the  use  of  the  system  would  be  required  for 
extracting  all  the  capabilities  of  the  system. 

The  system  was  also  thought  to  be  such  wearable  so  it  can  be 
used  as  underwear.  The  idea  is  to  make  the  user  feel  as 
comfortable  as  possible  with  this  electronic  aid,  in  such  a  way 
that  it  can  be  use  as  another  clothe.  Remember  that  the  user 
interface  works  with  vibrating  signals,  therefore  it  is 
recommended  not  to  use  thick  clothes  under  the  motor 
matrix. 

Nevertheless  the  validation  with  patients  must  be  the  next 
stage  for  the  project. 
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